Minimum Entropy Estimation of Hierarchical Random Graph Parameters for Character Recognition

2.1 Entropy and mutual information In this paper, we propose a new parameter estimation method called minimum entropy estimation (MEE), which tries to minimize the conditional entropy of the models given the input data. Since there is no assumption in MEE for the correctness of the parameter space of models, MEE will perform not less than the other estimation methods such as maximum likelihood estimation (MLE) and maximum mutual information estimation (MMIE), under the condition that the training data size is large enough. In the experiments, the three estimation methods are applied to the parameter estimation of hierarchical random graphs so that their estimation performance can be compared with each other.